{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Ch `12`: Concept `01`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Ranking by neural network"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Import the relevant libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "import random\n",
    "\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's fabricate some data. We'll call `get_data()` to generate two datasets: `data_a` and `data_b`.\n",
    "\n",
    "We'll use the convention that points in `data_a` are ranked lower than those in `data_b`. So we need to learn a ranking function (i.e. utility function) that scores points in `data_a` lower. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD8CAYAAACfF6SlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAF55JREFUeJzt3X+MHOV9x/H3x2ASGZDPyNcGYZ8PWksJKQnQk/MLBVMa\nMFTgRKlUUzc1CdE1EaQGqqiklsLFyE2USrWdggIWtQiSY6KSkJoISlx+lCrUqc/UYAMFHGIbWyi+\nYGxCHYUYf/vHzOK55fZ29m73Znfn85JWt/M8M7vfnRt9Z/Z5Zp9HEYGZmZXLtKIDMDOzqefkb2ZW\nQk7+ZmYl5ORvZlZCTv5mZiXk5G9mVkJO/mZmJeTkb2ZWQk7+ZmYldGLRAYxl9uzZ0d/fX3QYZmYd\nY9u2bb+MiN6867dl8u/v72d4eLjoMMzMOoakPY2s72YfM7MScvI3MyshJ38zsxJy8jczKyEnfzOz\nEnLyNzMrobrJX9JcSY9KelbSM5KWj7GOJH1L0i5JT0s6P1O3TNKL6WNZsz+AmXWo6lkEPavglMpz\n5X8U+JuIOBv4MHCtpLOr1rkMmJ8+BoFvA0g6DbgZ+BCwALhZ0qwmxW5mnWpoCG644XjCj0iWh4aK\njKpU6ib/iHglIp5Mn/8KeA44o2q1xcDdkdgC9Eg6HbgU2BwRByPiNWAzsKipn8DMOksEHDoEa9ce\nPwHccEOyfOiQvwFMkYZ+4SupHzgP+GlV1RnAy5nlfWlZrfKxXnuQ5FsDfX19jYRlZp1EgtWrk+dr\n1yYPgOXLk3KpuNhKJHeHr6RTgO8D10fE680OJCLWRcRARAz09uYensLMOlH2BFDhxD+lciV/SdNJ\nEv+GiPjBGKvsB+ZmluekZbXKzazMKk09Wdk+AGu5PHf7CPhn4LmI+Mcaq20C/jK96+fDwOGIeAV4\nCLhE0qy0o/eStMzMyirbxr98ORw7lvzN9gFYy+Vp8/8Y8Blgh6TtadnfAX0AEXE78ABwObALOAJ8\nNq07KOkWYGu63cqIONi88M2sLUSMbrKpXs6SoKdndBt/pQmop8dNP1NE0YZn2YGBgfCQzmYdYmgo\nuUunksgrV/Y9PePfutnICcPqkrQtIgbyru9f+JrZxE3mts3qRO/EP6XacjIXM+sQvm2zY7nZx8wm\nLwKmZRoSjh1z4p9ibvYxs6nl2zY7kpO/mU2cb9vsWG7zN7OJ822bHctt/mY2eb5ts3Bu8zezqefb\nNjuOk7+ZWQk5+ZuZlZCTv5lZCTn5m5mVkJO/mVkJOfmbmZWQk7/ZeKp/B9OGv4sxmwgnf7NahoZG\nD1FQGcpgvDHqzTqEk7/ZWCYzTr1ZB/DYPmZj8Tj11uU8to/ZeDxOvXWIpo/tI2m9pAOSdtao/7Kk\n7eljp6S3JJ2W1u2WtCOtcza3zuJx6m2iOuBGgTxt/ncBi2pVRsQ/RMS5EXEu8BXgPyLiYGaVi9L6\n3Gcks8J5nHqbqA65UaBum39EPC6pP+frXQVsnExAZm3B49TbRGRvFIDkmMleRLTRUNe52vzT5P+j\niPiDcdaZAewDfr9y5S/p58BrQAB3RMS6PEG5zd/ahsept0ZlvzVWTMGNAkWO538F8JOqJp8LIuJ8\n4DLgWkkfr7WxpEFJw5KGR0ZGmhiW2SR4nHprVPZbYkUb3iHWzOS/hKomn4jYn/49ANwHLKi1cUSs\ni4iBiBjo7e1tYlhmZlOoQ24UaErylzQTuBD410zZyZJOrTwHLgHGvGPIzKwrdNCNAnU7fCVtBBYC\nsyXtA24GpgNExO3pap8CfhwR/5fZ9HeB+5R81TkR+G5E/FvzQjczazMddKOAf+RlZtZsBdwo4Anc\nzcyK1gE3Cjj5m5mVkJO/mVkJOfmbmZWQk7+ZWQk5+ZuZlZCTv5lZCTn5m5mVkJO/mVkJOfmbmZWQ\nk7+ZWQk5+ZuZlZCTv5lZCTn5m5mVkJO/mVkJOfmbmZWQk7+ZWQk5+ZuZlZCTv5lZCdVN/pLWSzog\naWeN+oWSDkvanj6+mqlbJOl5Sbsk3dTMwM3MbOLyXPnfBSyqs85/RsS56WMlgKQTgNuAy4Czgask\nnT2ZYM3MrDnqJv+IeBw4OIHXXgDsioiXIuJN4B5g8QRex8zMmqxZbf4fkfSUpAclvT8tOwN4ObPO\nvrTMzMwKdmITXuNJYF5EvCHpcuCHwPxGX0TSIDAI0NfX14SwzMyslklf+UfE6xHxRvr8AWC6pNnA\nfmBuZtU5aVmt11kXEQMRMdDb2zvZsMzMbByTTv6S3iNJ6fMF6Wu+CmwF5ks6U9JJwBJg02Tfz8xq\niBh/2SyjbrOPpI3AQmC2pH3AzcB0gIi4HfhT4IuSjgK/BpZERABHJV0HPAScAKyPiGda8inMym5o\nCA4dgtWrQUoS/w03QE9PUmdWpW7yj4ir6tTfCtxao+4B4IGJhWZmuUQkiX/t2mR59eok8a9dC8uX\nJ/XJl3OztzWjw9esnKqTalFJVkoSPiQJv3ISWL78+DcBsyoe3sFsIoaGkqvrSrt6pZmlqCaW7Amg\nwonfxuHkb9aobDNL5QRQaWY5dKiYjtZKDFnZk5NZFSd/s0ZVrrKXL08S/rRpx9vXi7jazp58li+H\nY8eOx+YTgNXg5G82Ee3UzCIld/VkTz6Vk1NPj5t+bEzu8DWbiFrNLEWdAIaGRnc4V04ATvxWg6/8\nzRrVrs0s1Yneid/G4St/s0bVamYBN7NYx1C0YWfQwMBADA8PFx2G2fiq7+s/dizp/K1Vb9ZCkrZF\nxEDe9d3sYzZR2cQ+NAQ33tg+9/2b1eHkbzZZ7Xjfv1kdbvM3mywPr2AdyG3+Zs0SMbrN/9gxJ36b\nMm7zNyuCh1ewDuPkbzZZ7Xrfv9k43OZvNlm+7986kNv8zZqlXcb3t1Jym79ZUTy8gnUQJ38zm3qe\nbL5wdZO/pPWSDkjaWaN+qaSnJe2Q9ISkD2bqdqfl2yW5HcfM2m8WtJLKc+V/F7BonPqfAxdGxDnA\nLcC6qvqLIuLcRtqizKxL+dfQbaPu3T4R8bik/nHqn8gsbgHmTD4sM+tK/jV022h2m/81wIOZ5QB+\nLGmbpMEmv5eZdaJ2mgWtxJqW/CVdRJL8/zZTfEFEnA9cBlwr6ePjbD8oaVjS8MjISLPCMiuvdu1U\n9a+h20JTkr+kDwB3Aosj4tVKeUTsT/8eAO4DFtR6jYhYFxEDETHQ29vbjLDMyqtdO1X9a+i2Menk\nL6kP+AHwmYh4IVN+sqRTK8+BS4Ax7xgysyZq505VTzbfNur+wlfSRmAhMBv4BXAzMB0gIm6XdCfw\naWBPusnRiBiQdBbJ1T4kHcvfjYhVeYLyL3zNJimb8CvaqVPVv4ZuukZ/4evhHcy6lYeYLhUP72Bm\n7lS1upz8zbqNO1UtBw/pbNZtPMS05eA2f7Nu5U7VUnGbv5klPMS0jcPJ38yshJz8zcxKyMnfzKyE\nnPzNzErIyd/MrISc/M3MSsjJ38yshJz8zcxKyMnfzKyEnPzNzErIyd/MrISc/M3MSsjJ38ysSNUj\nK0/RSMtO/mZmRRkaGj3BTmUinqGhlr91ruQvab2kA5J21qiXpG9J2iXpaUnnZ+qWSXoxfSxrVuBm\nZh0tAg4dGj3DWmUGtkOHWv4NIO9MXncBtwJ316i/DJifPj4EfBv4kKTTgJuBASCAbZI2RcRrkwna\nzKzjZWdYW7s2ecDoGdhaKNeVf0Q8DhwcZ5XFwN2R2AL0SDoduBTYHBEH04S/GVg02aDNzLpC9gRQ\nMQWJH5rX5n8G8HJmeV9aVqvczMwqTT1Z2T6AFmqbDl9Jg5KGJQ2PjIwUHY6ZWWtl2/iXL4djx5K/\n2T6AFsrb5l/PfmBuZnlOWrYfWFhV/thYLxAR64B1kEzg3qS4zMzakwQ9PaPb+CtNQD09LW/6aVby\n3wRcJ+kekg7fwxHxiqSHgL+XNCtd7xLgK016TzOzzjY0lFzhVxJ95QQwBW3+uZK/pI0kV/CzJe0j\nuYNnOkBE3A48AFwO7AKOAJ9N6w5KugXYmr7UyogYr+PYzKxcqhP9FCR+yJn8I+KqOvUBXFujbj2w\nvvHQzMysVdqmw9fMzKaOk7+ZWQk5+du4NuzYQP+afqZ9bRr9a/rZsGND0SGZWRM0624f60Ibdmxg\n8P5Bjvz2CAB7Du9h8P5BAJaes7TI0MxsknzlbzWteHjF24m/4shvj7Di4RUFRWRmzeLk3wTd2jSy\n9/DehsrNrHM4+U9SpWlkz+E9BPF200g3nAD6ZvY1VG5mncPJf5K6uWlk1cWrmDF9xqiyGdNnsOri\nVQVFZGbN4uQ/Sd3cNLL0nKWsu2Id82bOQ4h5M+ex7op17uw16wK+22eS+mb2sefwnjHLu8HSc5Z2\nRLLfsGMDKx5ewd7De+mb2ceqi1d1RNxmRfGV/yS5aaR43dzvYtYqTv6T5KaR4nVzv4tZq7jZpwk6\npWmkW3Vzv4tZq/jK3zqeb0k1a1xXJf9u/bGVjc/9LmaN65rk706/8nK/i1njFFMwS3yjBgYGYnh4\nuKFt+tf0j3nL5byZ89h9/e4mRWZm1p4kbYuIgbzrd82Vvzv9zMzy65rk704/M7P8uib5d1unnzuv\nzayVciV/SYskPS9pl6SbxqhfLWl7+nhB0qFM3VuZuk3NDD6rmzr93HltZq1Wt8NX0gnAC8AngH3A\nVuCqiHi2xvpfAs6LiM+ly29ExCmNBDWRDt9u4s5rM2tUKzp8FwC7IuKliHgTuAdYPM76VwEb8wZg\n7+TOazNrtTzJ/wzg5czyvrTsHSTNA84EHskUv1vSsKQtkj5Z600kDabrDY+MjOQIq3u589rMWq3Z\nHb5LgHsj4q1M2bz0q8ifA2sk/d5YG0bEuogYiIiB3t7eJofVWbqt89rM2k+e5L8fmJtZnpOWjWUJ\nVU0+EbE//fsS8BhwXsNRlkw3dV6bWXvKM6rnVmC+pDNJkv4Skqv4USS9F5gF/FembBZwJCJ+I2k2\n8DHgm80IvNt5pFAza6W6yT8ijkq6DngIOAFYHxHPSFoJDEdE5fbNJcA9Mfr2ofcBd0g6RvIt4xu1\n7hIyM7Op0zVj+5iZlVlpx/YxM7P8SpP8PVyCmdlxpZjGsTJcQmWe18pwCYA7Vc2slEpx5e8Jvs3M\nRitF8vdwCWZmo5Ui+Xu4BDOz0UqR/D1cgpnZaKVI/h4uwcxsNP/Iy8ysC/hHXmZmVpeTv5lZCTn5\nm5mVkJO/mVkJOfmbmZWQk7+ZWQk5+ZuZlZCTv5lZCTn5j8NzAJhZtyrFeP4T4TkAzKyb5bryl7RI\n0vOSdkm6aYz6qyWNSNqePj6fqVsm6cX0sayZwbeS5wAws25W98pf0gnAbcAngH3AVkmbIuLZqlW/\nFxHXVW17GnAzMAAEsC3d9rWmRN9CngPAzLpZniv/BcCuiHgpIt4E7gEW53z9S4HNEXEwTfibgUUT\nC3VqeQ4AM+tmeZL/GcDLmeV9aVm1T0t6WtK9kuY2uC2SBiUNSxoeGRnJEVZreQ6AYrmz3ay1mnW3\nz/1Af0R8gOTq/juNvkBErIuIgYgY6O3tbVJYE+c5AIpT6Wzfc3gPQbzd2e4TgFnz5LnbZz8wN7M8\nJy17W0S8mlm8E/hmZtuFVds+1miQRVl6zlIn+wKM19nu/4dZc+S58t8KzJd0pqSTgCXApuwKkk7P\nLF4JPJc+fwi4RNIsSbOAS9Iys5rc2W7WenWv/CPiqKTrSJL2CcD6iHhG0kpgOCI2AX8t6UrgKHAQ\nuDrd9qCkW0hOIAArI+JgCz6HdZG+mX3sObxnzHIzaw5P42htp/oHdpB0trvPxaw2T+NoHc+d7Wat\n5yt/M7Mu4Ct/MzOry8nfzKyEnPzNzErIyd/MrISc/M3MSsjJ38yshJz8zcxKyMnfzKyEnPzNzErI\nyd/MrISc/M3MSsjJ38yshJz8zcxKyMnfzKyEnPzNzErIyb/LbNixgf41/Uz72jT61/SzYceGXHVm\nVi515/C1zlE9/eGew3sYvH/w7fpadZ4hy6x8cs3kJWkRsJZkAvc7I+IbVfU3Ap8nmcB9BPhcROxJ\n694CdqSr7o2IK+u9n2fympj+Nf1jTnw+b+Y8gJp1u6/f3erQzKzFGp3Jq+6Vv6QTgNuATwD7gK2S\nNkXEs5nV/gcYiIgjkr4IfBP4s7Tu1xFxbu5PYBO29/Dehsrr1ZlZ98rT5r8A2BURL0XEm8A9wOLs\nChHxaEQcSRe3AHOaG6bl0Tezr2b5eHVmVj55kv8ZwMuZ5X1pWS3XAA9mlt8taVjSFkmfnECMltOq\ni1cxY/qMUWUzps9g1cWrxq0zs/JpaoevpL8ABoALM8XzImK/pLOARyTtiIifjbHtIDAI0Nfnq9GJ\nqHTcrnh4BXsP76VvZh+rLl41qkN3vDozK4+6Hb6SPgIMRcSl6fJXACLi61Xr/THwT8CFEXGgxmvd\nBfwoIu4d7z3d4Wtm1phGO3zzNPtsBeZLOlPSScASYFPVm54H3AFcmU38kmZJelf6fDbwMSDbUWxm\nZgWo2+wTEUclXQc8RHKr5/qIeEbSSmA4IjYB/wCcAvyLJDh+S+f7gDskHSM50Xyj6i4hMzMrQK77\n/Keam33MzBrTimYfMzPrMk7+ZmYl5ORvZlZCTv5mZiXUlh2+kkaAd45CNnmzgV+24HVbpZPi7aRY\nwfG2UifFCt0T77yI6M37Im2Z/FtF0nAjveFF66R4OylWcLyt1EmxQnnjdbOPmVkJOfmbmZVQ2ZL/\nuqIDaFAnxdtJsYLjbaVOihVKGm+p2vzNzCxRtit/MzOjS5K/pEWSnpe0S9JNY9TfKOlZSU9LeljS\nvEzdW5K2p49N1dsWFO/VkkYycX0+U7dM0ovpY1mbxLs6E+sLkg5l6qZ0/0paL+mApJ016iXpW+ln\neVrS+Zm6IvZtvXiXpnHukPSEpA9m6nan5dsltXwwrByxLpR0OPP//mqmbtxjqKB4v5yJdWd6rJ6W\n1k3pvk3fc66kR9Nc9Yyk5WOs07zjNyI6+kEy0ujPgLOAk4CngLOr1rkImJE+/yLwvUzdG20Y79XA\nrWNsexrwUvp3Vvp8VtHxVq3/JZKRX4vavx8Hzgd21qi/nGSmOQEfBn5a1L7NGe9HK3EAl1XiTZd3\nA7PbaN8uJJmvY1LH0FTFW7XuFcAjRe3b9D1PB85Pn58KvDBGbmja8dsNV/6dNsdw3XjHcSmwOSIO\nRsRrwGZgUYvirGg03quAjS2OqaaIeBw4OM4qi4G7I7EF6JF0OsXs27rxRsQTaTxQ8LGbY9/WMplj\nfsIajLfQ4xYgIl6JiCfT578CnuOdU+Y27fjthuTfaXMM54330+nXunslzW1w22bK/Z5pc9qZwCOZ\n4nabw7nW5yli3zaq+tgN4MeStimZBrUdfETSU5IelPT+tKyt962kGSSJ8vuZ4kL3raR+4Dzgp1VV\nTTt+mzqHb7vTJOYYnmL3Axsj4jeS/gr4DvBHBceUxxLg3oh4K1PWjvu340i6iCT5X5ApviDdt78D\nbJb0v+nVblGeJPl/vyHpcuCHwPwC48nrCuAnEZH9llDYvpV0CsmJ6PqIeL1V79MNV/77gbmZ5Tlp\n2ShK5hheQTLV5G8q5RGxP/37EvAYydm2lerGGxGvZmK8E/jDvNu2QCPvuYSqr84F7N96an2eIvZt\nLpI+QHIcLI6IVyvlmX17ALiPpHmlMBHxekS8kT5/AJiuZPrWtt23qfGO2yndt5KmkyT+DRHxgzFW\nad7xO5UdGi3qJDmRpHPjTI53Jr2/ap3zSDqc5leVzwLelT6fDbxIizuicsZ7eub5p4AtcbxT5+dp\n3LPS56cVHW+63ntJOslU5P5N36uf2p2Sf8LoDrP/Lmrf5oy3D9gFfLSq/GTg1MzzJ4BFBcf6nsr/\nnyRZ7k33c65jaKrjTetnkvQLnNwG+1bA3cCacdZp2vHb8p0/Rf/gy0l6xn8GrEjLVpJc5QP8O/AL\nYHv62JSWfxTYkR6MO4Br2iTerwPPpHE9Crw3s+3n0mSwC/hsO8SbLg+RzNGc3W7K9y/JFdwrwG9J\n2j2vAb4AfCGtF3Bb+ll2AAMF79t68d4JvJY5dofT8rPS/fpUeqysaINYr8sct1vInLDGOoaKjjdd\n52rgnqrtpnzfpu97AUlfw9OZ//flrTp+/QtfM7MS6oY2fzMza5CTv5lZCTn5m5mVkJO/mVkJOfmb\nmZWQk7+ZWQk5+ZuZlZCTv5lZCf0/ZKBlx7cgWDwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f12eb2e8860>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "n_features = 2\n",
    "\n",
    "def get_data():\n",
    "    data_a = np.random.rand(10, n_features) + 1\n",
    "    data_b = np.random.rand(10, n_features)\n",
    "    \n",
    "    plt.scatter(data_a[:, 0], data_a[:, 1], c='r', marker='x')\n",
    "    plt.scatter(data_b[:, 0], data_b[:, 1], c='g', marker='o')\n",
    "    plt.show()\n",
    "    \n",
    "    return data_a, data_b\n",
    "\n",
    "def get_data2():\n",
    "    data_a = np.asarray([[0.1, 0.9], [0.1, 0.8]])\n",
    "    data_b = np.asarray([[0.4,0.05], [0.45, 0.1]])\n",
    "    \n",
    "    plt.scatter(data_a[:, 0], data_a[:, 1], c='r', marker='x')\n",
    "    plt.scatter(data_b[:, 0], data_b[:, 1], c='g', marker='o')\n",
    "    plt.xlim([0, 0.5])\n",
    "    plt.ylim([0, 1])\n",
    "    plt.axes().set_aspect('equal')\n",
    "    plt.show()\n",
    "    \n",
    "    \n",
    "    return data_a, data_b\n",
    "\n",
    "data_a, data_b = get_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's define our ranking model. It'll take in two items (`x1` and `x2`), and return a score (`s1` and `s2`) for each item. \n",
    "\n",
    "Our model introduces a hyper-parameter called `n_hidden` to tweak the number of neurons in the hidden layer of the network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "n_hidden = 10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When defining the model, let's organize it into separate scopes. That way, the TensorBoard visualization will look very clean."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "with tf.name_scope(\"input\"):\n",
    "    x1 = tf.placeholder(tf.float32, [None, n_features], name=\"x1\")\n",
    "    x2 = tf.placeholder(tf.float32, [None, n_features], name=\"x2\")\n",
    "    dropout_keep_prob = tf.placeholder(tf.float32, name='dropout_prob')\n",
    "\n",
    "\n",
    "with tf.name_scope(\"hidden_layer\"):\n",
    "    with tf.name_scope(\"weights\"):\n",
    "        w1 = tf.Variable(tf.random_normal([n_features, n_hidden]), name=\"w1\")\n",
    "        tf.summary.histogram(\"w1\", w1)\n",
    "        b1 = tf.Variable(tf.random_normal([n_hidden]), name=\"b1\")\n",
    "        tf.summary.histogram(\"b1\", b1)\n",
    " \n",
    "    with tf.name_scope(\"output\"):\n",
    "        h1 = tf.nn.dropout(tf.nn.relu(tf.matmul(x1,w1) + b1), keep_prob=dropout_keep_prob)\n",
    "        tf.summary.histogram(\"h1\", h1)\n",
    "        h2 = tf.nn.dropout(tf.nn.relu(tf.matmul(x2, w1) + b1), keep_prob=dropout_keep_prob)\n",
    "        tf.summary.histogram(\"h2\", h2)\n",
    " \n",
    "\n",
    "with tf.name_scope(\"output_layer\"):\n",
    "    with tf.name_scope(\"weights\"):\n",
    "        w2 = tf.Variable(tf.random_normal([n_hidden, 1]), name=\"w2\")\n",
    "        tf.summary.histogram(\"w2\", w2)\n",
    "        b2 = tf.Variable(tf.random_normal([1]), name=\"b2\")\n",
    "        tf.summary.histogram(\"b2\", b2)\n",
    " \n",
    "    with tf.name_scope(\"output\"):\n",
    "        s1 = tf.matmul(h1, w2) + b2\n",
    "        s2 = tf.matmul(h2, w2) + b2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The loss function will involve comparing `s1` and `s2`. \n",
    "\n",
    "Since we're trying to acheive the inequality `Score(x1) < Score(x2)`, we need the loss function to insinuate `s1 < s2`. \n",
    "\n",
    "In other words, the loss function tries to guarantee that `s1 - s2 < 0`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "with tf.name_scope(\"loss\"):\n",
    "    s12 = s1 - s2\n",
    "    s12_flat = tf.reshape(s12, [-1])\n",
    "    \n",
    "    pred = tf.sigmoid(s12)\n",
    "    lable_p = tf.sigmoid(-tf.ones_like(s12))\n",
    "    \n",
    "    cross_entropy = tf.nn.softmax_cross_entropy_with_logits(labels=tf.zeros_like(s12_flat), logits=s12_flat + 1)\n",
    "    \n",
    "    loss = tf.reduce_mean(cross_entropy)\n",
    "    tf.summary.scalar(\"loss\", loss)\n",
    " \n",
    "with tf.name_scope(\"train_op\"):\n",
    "    train_op = tf.train.AdamOptimizer(0.001).minimize(loss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Start the session and prepare peripheral ops."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "sess = tf.InteractiveSession()\n",
    "summary_op = tf.summary.merge_all()\n",
    "writer = tf.summary.FileWriter(\"tb_files\", sess.graph)\n",
    "init = tf.global_variables_initializer()\n",
    "sess.run(init)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Train the model with the training data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "for epoch in range(0, 10000):\n",
    "    loss_val, _ = sess.run([loss, train_op], feed_dict={x1:data_a, x2:data_b, dropout_keep_prob:0.5})\n",
    "    if epoch % 100 == 0 :\n",
    "        summary_result = sess.run(summary_op, feed_dict={x1:data_a, x2:data_b, dropout_keep_prob:1})\n",
    "        writer.add_summary(summary_result, epoch)\n",
    "#         print(\"Epoch {}: Loss {}\".format(epoch, loss_val))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Visualize the results on a grid by accumulating a list of points to test."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "grid_size = 10\n",
    "data_test = []\n",
    "for y in np.linspace(0., 1., num=grid_size):\n",
    "    for x in np.linspace(0., 1., num=grid_size):\n",
    "        data_test.append([x, y])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the model on all the test points and visualize the utility scores of each point by a color."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "\n",
    "def visualize_results(data_test):\n",
    "    plt.figure()\n",
    "    scores_test = sess.run(s1, feed_dict={x1:data_test, dropout_keep_prob:1})\n",
    "    scores_img = np.reshape(scores_test, [grid_size, grid_size])\n",
    "    plt.imshow(scores_img, origin='lower')\n",
    "    plt.colorbar()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAASQAAAD8CAYAAADe49kaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEKZJREFUeJzt3V+I5eV9x/HPZ86ZdXdnV924NvVvdgvBIEIxDK2JEFJN\nqflDclOKgiENhb1powmBYArF216EkFyUwGCSXiiGYoSKBDWYeJGbbdY/NOomRNTomrXuauuadXV2\n5nx6cc7IaNyZ35x5fnOes7/3S3648+85X92Zzzy/58/vcRIBQA1mJl0AAKwgkABUg0ACUA0CCUA1\nCCQA1SCQAFSDQAJQDQIJwKbZ/oHtV2w/uep9H7D9U9u/Hf17z3rtEEgASvh3STe85323SXo4yYcl\nPTx6e01uY6V2b/dc+hesG4YbF5dvU5IG5Zt0WwvgW6hVktxSuzNLLbW7WP5/8MxiO8Xm7cXibb6l\nk1rM25v6gfibv5rLq68tN/rcR//77QeTvDdw3sX2Pkn3J7lq9PZvJH0yyVHbF0l6JMkVa7XRb1TN\nBvUv2KM//Zdbirfrt9vp0M28VT7oepv7Xjlzuy3UKkn9N1tpVttfbSeZdx0p/0O+/fnXircpScvP\nPFe8zYN5eNNtvPrasv7rwcsbfW7vot9+xPahVe9aSLKwzpd9MMnR0Z9flvTB9V6nlUACUL9IGjTv\nch9PMj/2ayWx179vIJCAjoqi02l2yzam/7F90apbtlfW+wIGtYEOGzT8Z0z3SfrS6M9fkvSf630B\nPSSgo6JoudCklu27JX1S0l7bRyTdLulfJf2H7X+Q9DtJf7deOwQS0GEDlQmkJDed4UPXb6QdAgno\nqEhaLhRIpRBIQIeV6iGVQiABHRVJpyt7hDWBBHRUFG7ZAFQi0nJdeUQgAV01XKldFwIJ6CxrWS1t\nWB8TgQR01HBQm0ACUIHhOiQCCUAlBvSQANSAHhKAakTWcmUP/CCQgA7jlg1AFSJrMb1Jl/EuBBLQ\nUcOFkR24ZXMvmt1V/iHsS7Pt5Odyv/xvicFsO3/Rg5Z+hQxm22q3nVuCQX9b8TbT31u8TUna0cL3\nrZ/9RZF2GNQGUIXEWk4HekgApsOAHhKAGgwHteuKgLqqAbBlOjOoDWA6LLMOCUANalyp3aga21+z\n/ZTtJ23fbXt724UBaN8gM42urbLuK9m+RNItkuaTXCWpJ+nGtgsD0K7h5tqZRtdWaXrL1pe0w/Zp\nSTsl/b69kgBshcg6PW1bR5K8ZPtbkl6QdErSQ0kear0yAK1KVN3CyCa3bHskfUHSfkkXS5qzffP7\nfN4B24dsH1o+cbJ8pQAKswYNr63SJB4/Jem5JMeSnJZ0r6SPv/eTkiwkmU8y3zt3rnSdAAqLhj2k\nJtdWafJKL0i6xvZO25Z0vaTD7ZYFYCuUGtQuNRO/7islOSjpHkmPSfrV6GsWxnkxAPWIrEGaXWsp\nORPfaJYtye2Sbh/nBQDUaXgMUrG10UVm4usaYgewhYYHRTa51pLkJUkrM/FHJb0+7kw8gQR0VLSh\nldp7V2bRR9eBlXaazsQ3wV42oMM28MTI40nmz/Cxd2biJcn2ykz8nRuth0ACOipxqX1q78zEa7h4\n+npJh8ZpiEACOmo4qL35rSNJDtpemYlfkvS4xpyJJ5CAzir3TO1SM/GtBNLMzEC7d50q3u6p2fIn\nTUjS4mz5IzeWZ9vZtLjU1mkmrbXbSrMa9MtvZ0ivnd/Pg/6e8m2+VKBnIw6KBFCR2h7QRiABHbWy\nUrsmBBLQYTzkH0AVEun0gEACUIHhLRuBBKASG1ipvSUIJKCjmPYHUBFu2QBUZCufl90EgQR01HCW\nbcqOQQJwdmJhJICqcMsGoArMsgGoCrNsAKqQWEsEEoBacMsGoAqMIQGoCoEEoAqsQwJQlU6sQ+rP\nDHTh3Mni7b4xu1S8TUn6Q/+c4m2+1W/n6fanZ9t6EH07WwjS1uEB/fLtpoWDAyRp0C//d7a8bfO1\nJtISD2gDUAtu2QBUgTEkAFUJgQSgFp0Y1AZQv4QxJADVsJaZZQNQi9rGkBrFo+3zbd9j+9e2D9v+\nWNuFAWjXyl62Jtd6SmVE0x7SdyU9kORvbW+TtHOcFwNQkQzHkQopkhHrBpLt8yR9QtLfS1KSRUmL\n47wYgLqUmGUrmRFNbtn2Szom6Ye2H7d9h+259ynqgO1Dtg+dfv3UOLUA2EIZDWo3uSTtXfn5Hl0H\nVjXVKCOaaBJIfUkflfS9JFdLOinptj/6j0sWkswnmZ89b8c4tQDYYkmzS9LxlZ/v0bWwqplGGdFE\nk0A6IulIkoOjt+8ZvTiAKZe40bWOYhmxbiAleVnSi7avGL3reklPj/NiAOox7P1sPpBKZkTTWbav\nSLprNHr+rKQvj/NiAOpScKV2kYxoFEhJnpA0P84LAKhXqWn/UhnBSm2goyJrwNYRALUoty6yDAIJ\n6KrUt5eNQAK6rLIuEoEEdFgnekjbZpZ10c4Txdvd2d9evE1J2t4vf5rJG7PlTzKRpDdnt7XS7tst\nnZKyNNvOaSbLLbSbFk4ykaTBbPkf+kGBb4NIGgw6EEgApkAkdaGHBGA6FHz8SBEEEtBlBBKAOjTa\nOLulCCSgy+ghAahCpDDLBqAeBBKAWnDLBqAaBBKAKrAwEkBNWBgJoB7MsgGohekhAahCxKA2gFqY\nQW0AFaGHBKAag0kX8G4EEtBVrEMCUBNm2QDUo7JAquvYSgCd1tKpI0u6fMdrxdt9tbereJuStL1X\n/tSRHf3TxduUpBMtnbzyh347p5mcaumUlMXZ8t+6g147NwynZ8v/3k+hQ1dqu2WjhwR0VTTcOtLk\nasB2z/bjtu8ftyQCCeiyNLyauVXS4c2UQyABHeY0u9Ztx75U0mcl3bGZeggkoMvK9ZC+I+kb2uRS\nSwIJ6LLmgbTX9qFV14GVJmx/TtIrSR7dbDmsQwI6qunt2MjxJPNn+Ni1kj5v+zOStks61/adSW7e\naE2Ne0glRtABVKbALFuSbya5NMk+STdK+tk4YSRtrIe0MoJ+7jgvBKA+U7kOqdQIOoDKlJ32V5JH\nknxu3HKa9pBWRtB3j/tCACqzsTGkLbFuD6npCLrtAysj8Cf/d7FYgQBaVLiHtFlNbtlWRtCfl/Qj\nSdfZvvO9n5RkIcl8kvm5Pe3sXwJQlgfNrq2ybiCVHEEHgLWwDgnossrGkDYUSEkekfRIK5UA2FoV\nDmrTQwK6jEACUA0CCUANrK2dQWuCQAK6ijEkAFUhkABUowuBtM1L2n/OseLt7uq9VbxNSZrrl9+i\n99riXPE2JWlnv51tOa/3d7TS7onZ8ie6SNLJFk4zeavXzg6DpRZOSNFMmSThlg1APQgkAFUIs2wA\nakIPCUAtGEMCUA8CCUAVtvjha00QSEBHWdyyAagIgQSgHgQSgGoQSACqwG5/AFUhkADUgq0jAKrB\nLRuAOlS4MLLJybUAzlYFjtK2fZntn9t+2vZTtm8dtxx6SEBHFVypvSTp60kes71b0qO2f5rk6Y02\nRCABHebB5hMpyVFJR0d/fsP2YUmXSCKQADTUwhiS7X2SrpZ0cJyvJ5CADtvALdte24dWvb2QZOFd\nbdm7JP1Y0leTnBinHgIJ6LLmgXQ8yfyZPmh7VsMwuivJveOW00ogneMl7Zs9XrzdnTNvF29TknbP\nlD/NZHdLJ6Qc7+9qpd25lk4z2dHf2Uq7r/e3F2/zRG+5eJuSdKqFE1LcK7OiscSgtm1L+r6kw0m+\nvZm2mPYHuqzAtL+kayV9UdJ1tp8YXZ8Zpxxu2YCuKnTqSJJfaLiKYNMIJKCjeGIkgLqkrkQikIAO\no4cEoA7TuLm25MY5AHXxoNm1VZr0kIptnANQl6l7QFvJjXMAKhJN96D2WhvnbB+QdECS/uRihqaA\naVDboHbjldrrbZxLspBkPsn8+R/olawRQFvKrNQuplFXptTGOQD1mMqFkSU3zgGoSFLkAW0lNbll\nK7ZxDkBlpu2WreTGOQB1mbpbNgBnqUiq7JaNQAK6rK48IpCALuOWDUA1aptlI5CArqpwt39LD/lf\n1p/NjnUKypraeBi/JJ07RQ/539XW4QG93a20u6N3upV2d7ZwKMGO/o7ibUrS67PlDyQ4MrP5XbHD\nhZF1JRI9JKDLpm23P4CzFz0kAHXoyhgSgGlQ3142AgnoMm7ZAFSh0EGRJRFIQJfRQwJQjbryiEAC\nusyDuu7ZCCSgq6LqFkY2fsg/gLOLFTnNrnXbsm+w/Rvbz9i+bdyaCCSgy5Jm1xps9yT9m6RPS7pS\n0k22rxynHAIJ6LICgSTpLyQ9k+TZJIuSfiTpC+OUQyABXbUyhtTkWtslkl5c9faR0fs2jEFtoMM2\nMMu21/ahVW8vJFkoXQ+BBHRWo9uxFceTzJ/hYy9JumzV25eO3rdh3LIBXRWVGkP6paQP295ve5uk\nGyXdN05J9JCALiuwDinJku1/kvSgpJ6kHyR5apy2CCSgw0o9oC3JTyT9ZLPtEEhAl7G5FkAVEmm5\nrr0jrQTSNvd0eX9X8XZ3+83ibUrS7pnXWmjzVPE22223ndNM2jp9Za5X/vtrroWTTCRpewsnrxwu\ncOqIJHpIACpCIAGoQiTxTG0AdYiUDowhAZgCUTcGtQFMCcaQAFSjskBqtJet1NPgANSk4T62LQyt\ndXtIq54G99caPufkl7bvS/J028UBaFEkVfaQ/yY9pGJPgwNQmWnrIen9nwb3l+2UA2DrnMVbR2wf\nkHRAki6/hLFyoHqRUtk6pCa3bI2eBpdkIcl8kvkLL+iVqg9AmwZpdm2RJoFU7GlwACozbWNIJZ8G\nB6AiSXWzbI0Ge0o9DQ5AZSpbGMnoM9BZUZaXJ13EuxBIQFfx+BEAVals2p9AAjoqkkIPCUAVwgPa\nAFSktkFtp4VpP9vHJP2uwafulXS8eAHtmaZ6p6lWabrqraHWDyW5cDMN2H5Aw/+WJo4nuWEzr9dE\nK4HU+MXtQ0nmJ1bABk1TvdNUqzRd9U5TrdOm0QPaAGArEEgAqjHpQFqY8Otv1DTVO021StNV7zTV\nOlUmOoYEAKtNuocEAO+YWCBNy0kmti+z/XPbT9t+yvatk66pCds924/bvn/StazF9vm277H9a9uH\nbX9s0jWtxfbXRt8HT9q+2/b2Sdd0NplIIK06yeTTkq6UdJPtKydRSwNLkr6e5EpJ10j6x4prXe1W\nSYcnXUQD35X0QJKPSPpzVVyz7Usk3SJpPslVGj4f7MbJVnV2mVQPaWpOMklyNMljoz+/oeEPzCWT\nrWptti+V9FlJd0y6lrXYPk/SJyR9X5KSLCb5v8lWta6+pB22+5J2Svr9hOs5q0wqkN7vJJOqf8gl\nyfY+SVdLOjjZStb1HUnfkFTXRqU/tl/SMUk/HN1e3mF7btJFnUmSlyR9S9ILko5Kej3JQ5Ot6uzC\noHZDtndJ+rGkryY5Mel6zsT25yS9kuTRSdfSQF/SRyV9L8nVkk5Kqnk8cY+GPfn9ki6WNGf75slW\ndXaZVCA1OsmkFrZnNQyju5LcO+l61nGtpM/bfl7DW+HrbN852ZLO6IikI0lWepz3aBhQtfqUpOeS\nHEtyWtK9kj4+4ZrOKpMKpKk5ycS2NRzjOJzk25OuZz1Jvpnk0iT7NPz/+rMkVf4WT/KypBdtXzF6\n1/WSaj6i/QVJ19jeOfq+uF4VD8JPo4k8fmTKTjK5VtIXJf3K9hOj9/3z6OADbN5XJN01+sX0rKQv\nT7ieM0py0PY9kh7TcPb1cbFquyhWagOoBoPaAKpBIAGoBoEEoBoEEoBqEEgAqkEgAagGgQSgGgQS\ngGr8P9OMdTWO/bgzAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f12e8deadd8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "visualize_results(data_test)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
